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Many  weapon  system  failures  are  attributed  to  premature  transfer  of 
technology  to  operational  systems.  Insufficient  measures  of  assessing 
technology  readiness  are  major  contributors  to  such  failures.  This  paper 
presents  a methodology  to  measure  the  performance  risk  of  technology 
in  order  to  determine  its  transition  readiness.  This  methodology  is  referred 
to  as  Technology  Performance  Risk  Index  (TPRI).  The  TPRI  can  track 
technology  readiness  through  a life  cycle,  or  it  can  be  used  at  a specific 
time  to  support  a particular  system  milestone  decision.  The  TPRI  is  computed 
using  the  performance  requirements,  the  Degree  of  Difficulty  (DD),  and 
the  unmet  performance.  These  components  are  combined  in  a closed- 
loop  feedback  manner  to  analytically  calculate  the  performance  risk.  TPRI 
is  illustrated  by  an  example  using  published  system  requirements  data. 


Since  World  War  II,  the  United  States  Armed  Forces  have  maintained  a 
technological  advantage  over  adversaries.  Today,  the  military  is  facing  continued 
threats  that  require  an  accelerated  pace  of  technology  development  amid  global 
proliferation  of  military  technologies  (Lukens,  2003).  The  Department  of  Defense 
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Changing  to  a capability-based  acquisition  strategy  is  another  indicator  of  the 
significance  of  technology.  Mr.  Pete  Aldridge  identified  in  a memo  (Aldridge,  2002) 
to  the  Secretary  of  Defense  his  intent  to  accelerate  the  flow  of  technology  to  the 
warfighter.  Later,  Mr.  Aldridge  (2003)  announced  his  goal  to  initiate  high-leverage 
technologies  to  create  the  warfighting  capabilities  and  strategies  of  the  future.  There- 
fore, it  is  imperative  that  the  technology  meets  maturity  and  performance  requirements, 
prior  to  being  transitioned  to  the  acquisition  system.  Furthermore,  a capability-based 
acquisition  strategy  will  allow  acquisition  programs  to  pull  advanced  technology  into 
systems  faster,  thus  fielding  systems  with  advanced  technologies  to  the  warfighter 
faster.  The  capability-based  acquisition  cycle  also  means  that  requirements  will  evolve 
faster,  which  mandates  close  monitoring  of  systems’  requirements  and  the  technologies 
we  use  to  meet  these  requirements. 

THE  PROBLEM 

Development  of  new  defense  technologies  within  the  DoD  is  a multi-dimensional 
problem.  First,  DoD  must  resolve  issues  that  result  from  immature  technologies 
transition.  The  General  Accounting  Office  (GAO)  has  stated  that  immature  technology 
transition  is  the  leading  cause  of  weapon  system  problems  (GAO,  1999).  An  important 
factor  in  the  success  of  a new  weapon  system  is  ensuring  that  technologies  are  mature 
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prior  to  being  integrated  (GAO,  1999).  Second,  the  creation  of  parallel  paths  for 
the  development  of  technology  and  the  development  of  an  acquisition  weapon 
system  has  diluted  the  link  between  technology  and  system  performance  require- 
ments. The  technologist  has  responsibility  for  managing  the  development  of  the 
technology,  while  the  weapon  system  acquisitionist  has  responsibility  for  the 
development  of  the  weapon  system.  Unfortunately,  the  technologist  has  different 
goals,  environments,  and  perspectives  than  the  system  acquisitionist  (McGrath, 
2003).  The  original  reasoning  behind  this  deliberate  separation  is  that  it  allows 
the  acquisitionist  to  focus  on  meeting  requirements  for  the  system  development, 
while  providing  the  technologist  an  environment  to  explore  capabilities  of  the 
technology.  An  unforeseen  result  of  this  separation  is  that  two  conflicting  drives 
of  motivation  are  generated. 

The  technologist  is  motivated  to  transition  technologies  into  weapon  systems.  Thus, 
technologists  are  optimistic  on  the  maturity  assessment  of  their  technology.  The 
acquisitionist  is  motivated  to  meet  system  requirements,  and  often  uses  a risk  adverse 
approach  for  the  design  process.  Consequently,  the  acquisitionist  is  more  likely  to 
underestimate  the  maturity  of  new  technologies.  This  forces  the  technologist  to  focus 
on  risk  mitigation.  These  conflicting  motivations  justify  the  need  for  an  objective 
methodology  to  assess  a technologies  fit  with  system  performance  requirements.  As 
a technology’s  maturity  increases,  the  criticality  for  decision  support  tools  to  deter- 
mine transition  readiness  is  also  increased.  Hence,  a common  understanding,  between 
the  technologist  and  the  acquisitionist,  is  needed. 


PROPOSED  METHODOLOGY 

One  approach  to  develop  a common  understanding  of  technology  readiness  is  to 
utilize  a modified  version  of  Garvey’s  system  performance  risk  index  (Garvey  & 
Cho,  2003).  The  threshold  value  of  a Technical  Performance  Measure  (TPM)  divides 
performance  into  acceptable  and  unacceptable  risk  regions.  In  this  manner,  it  is  the 
goal  of  a system  developer  to  reach  the  acceptable  performance  risk  region.  To  get 
into  the  acceptable  performance  risk  region,  the  technology  must  meet  or  exceed  the 
identified  TPM  threshold.  Garvey  provided  guidelines  for  calculating  the  achieved 
performance. 

The  proposed  methodology  to  assess  technology  readiness  proposed  in  this  paper 
is  referred  to  as  the  Technology  Performance  Risk  Index  (TPRI).  The  index  is  based 
on  the  system’s  performance  requirements  and  the  ability  of  the  technology  to  achieve 
that  performance.  The  achieved  performance  is  normalized  so  multiple  requirements 
can  be  assessed  simultaneously.  A condensed  solution  for  the  two  cases  of  the  re- 
quired performance  behaviors  is  then  calculated.  In  the  first  case,  performance  must 
decrease  to  meet  the  established  TPM  threshold.  The  achieved  performance  is  com- 
puted as  the  inverse  of  the  percentage  of  the  TPM  threshold  that  the  measured  per- 
formance represents.  The  achieved  performance,  A.^  at  time  i,  for  TPM  j is  calculated 
by: 
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threshold 


Equation  1 


where  is  the  measured  performance.  Examples  of  this  type  of  required  per- 
formance behavior  are  weight  constraints  or  mean  time  to  repair  TPMs.  In  the 
second  case,  performance  must  increase  to  meet  the  threshold  and  Aip  at  time 
i,  for  the  jth  TPM  is  computed  using: 
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Equation  2 


Again,  nty  is  the  measured  performance.  In  the  increasing  performance  behavior, 
the  achieved  performance  is  equivalent  to  the  percentage  that  the  measured  performance 
represents  of  the  TPM  threshold.  Two  examples  of  increase  perfoimance  behavior 
TPMs  would  include  number  of  units  available  or  mean-time-between-failures. 

Garvey  defines  the  system  performance  risk  index  as  the  amount  of  the  remaining 
unmet  performance  relative  to  meeting  a set  of  identified  TPMs.  Emphasis  is  placed 
on  the  unmet  performance  as  issues  for  management  to  focus  and  resolve  with  pri- 
orities and  allocation  of  resources.  Although  Garvey’s  method  provides  a quantitative 
measure  of  meeting  a set  of  established  requirements,  it  inherently  lacks  the  inclusion 
of  a true  risk  measure.  More  specifically,  in  Garvey’s  approach,  unmet  performance 
does  not  indicate  what  it  will  take  to  reach  the  acceptable  risk  region.  It  merely  provides 
a measure  of  progress  achieved  at  a certain  time. 


Garvey  defines  the  system  performance  risk 
index  as  the  amount  of  the  remaining 
unmet  performance  relative  to  meeting 
a set  of  identified  TPMs . 


This  article  presents  a research  effort  to  address  the  linkage  between  technolo- 
gies, system  performance  requirements  and  risks  in  the  DoD  environment.  This 
methodology  is  used  to  measure  risk  associated  with  satisfying  an  identified  set 
of  TPMs  for  a given  system.  The  TPRI  provides  information  regarding  perfor- 
mance risk  associated  with  a technology  to  support  the  decisions  of  whether  or  not 
a certain  technology  is  ready  to  be  transitioned  to  an  acquisition  system.  The  TPRI 
offers  a measure  of  the  actual  performance  risk  of  a technology  by  providing  an 
assessment  of  a given  technologies  ability  to  achieve  performance  requirements. 
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Thus,  a common  understanding  of  performance  risks  between  technologist  and 
acquisitionist  is  accomplished.  The  impact  of  the  TPRI  can  be  described  using 
the  following  analogy:  a thermostat  measures  actual  outside  temperature,  yet  to 
a human,  the  wind  chill  factor  is  of  more  concern.  In  this  same  manner,  the  TPRI 
is  like  the  wind  chill  factor,  as  it  affords  valuable  information  that  allows  insight 
into  the  technology  that  would  not  be  obtained  from  individual  metrics. 

Performance-based  requirements  criteria  are  typically  established  by  an 
acquisitionist.  Typically,  these  criteria  and  the  basis  for  measurement  are  identified 
in  an  agreement  with  the  technologist.  This  serves  as  a basis  consisting  of  the  indi- 
vidual requirements  and  associated  threshold  values  against  which  technologies  will 
be  evaluated.  This  information  is  utilized  to  compare  against  the  measured  level  of 
performance  achieved. 

The  TPRI  extends  Garvey’s  methodology  to  include  a measure  of  performance  risk 
so  that  acquisitionists  can  have  access  to  information  that  will  assist  in  setting  priorities 
and  allocating  resources  (Garvey  & Cho,  2003).  For  this  purpose,  the  Degree  of 
Difficulty  (DD)  is  the  metric  utilized  by  TPRI.  The  DD  metric  (Mankins,  1998  & 
2002)  provides  a measure  of  anticipated  risk,  ranging  from  low  level  risk  to  high 
level  risk,  and  can  be  considered  as  a probability  of  failure  in  regards  to  the  technology 
achieving  the  objectives. 

In  TPRI,  the  DD  metric  is  modified  to  assign  a numerical  value  to  each  of 
Mankins’  defined  levels.  This  provides  a quantitative  means  to  combine  risks 
across  various  requirements.  In  this  context,  the  DD  metric  is  bounded  by  zero 
and  one.  Table  1 provides  a summary  of  each  of  the  DD  levels  with  anticipated 
risk,  and  the  numerical  value,  as  well  as  the  theoretical  boundary  levels.  The 
lower  boundary  with  a DD  of  zero  indicates  there  is  no  risk  involved  in  meeting 
the  objectives  and  is  guaranteed  success.  The  DD  at  level  1 has  a very  low  risk 
and  is  assigned  a 0.1  value;  a DD  at  level  2 has  an  anticipated  moderate  risk 
and  has  an  assigned  value  of  0.3,  while  the  DD  at  level  3 has  a high  risk,  with 


TABLE  1.  DEGREE  OF  DIFFICULTY  (DD)  LEVELS  WITH  RISKS 
AND  ASSIGNED  NUMERICAL  VALUES 


Degree  of  Difficulty 

Risk  Level 

DD  Value 

Level  0 

(Theoretical  Lower  Bound) 

No  Risk;  Guaranteed  Success 

0.0 

Level  1 

Very  Low  Risk 

0.1 

Level  2 

Moderate 

0.3 

Level  3 

High 

0.5 

Level  4 

Very  High 

0.7 

Level  5 

Extremely  High  Risk,  Requiring  a Fundamental  Breakthrough 

0.9 

Level  6 

(Theoretical  Upper  Bound) 

Guaranteed  Failure 

1.0 
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a 0.5  value.  A technology  with  very  high  risk  is  identified  as  DD  level  4 and 
has  a 0.7  value.  In  the  case  that  a technology  has  a high  expectation  of  failure, 
requiring  a fundamental  breakthrough,  is  assigned  a DD  level  5 with  a 0.9 
numerical  value.  In  addition,  the  theoretical  upper  bound  of  the  DD  metric 
indicates  the  highest  level  of  risk  with  no  anticipated  success  (guaranteed  fail- 
ure), and  is  assigned  a value  of  1. 

A major  attribute  of  TPRI  is  that  it  accounts  for  unmet  performance  and  associated 
risk  to  quantify  effects  upon  the  achieved  performance.  Risk  is  defined  as  a measure 
of  the  probability  and  the  severity  of  adverse  effects  (Haimes,  1998).  The  TPRI  is 
applicable  to  the  technical  risk  form;  referred  to  as  performance  risk.  The  TPMs 
provide  a gauge  of  technical  progress  measured  against  satisfying  an  identified 
set  of  thresholds  pertaining  to  performance  requirements.  The  TPRI  correlates 
the  performance  risk  associated  with  a technology  not  meeting  the  threshold  value 
of  a TPM. 


TPRI  METHODOLOGY 

To  formulate  the  TPRI  model,  the  unmet  performance  and  DD  are  combined  to 
adjust  the  achieved  performance.  The  TPRI  is  expressed  mathematically  as: 


TPRI  = 1 - - 


1 + (1  -A)*DD 


Equation  3 


The  unmet  performance,  1-A,  is  a measure  of  the  severity  component  of  risk,  while 
DD  signifies  the  probability  component  of  risk.  These  two  components  are  combined 
to  formulate  a measure  of  performance  risk  of  the  unmet  performance-based  on  a 
closed  loop  feedback  mechanism.  This  index  provides  a measure  of  performance  risks 
to  meet  the  TPM  threshold  in  an  effort  to  reach  the  acceptable  performance  risk 
region  goal. 


A major  attribute  of  TPRI  is  that  it  accounts  for 
unmet  performante  and  associated  risk  to 
quantify  effects  upon  the  achieved  performance . 


The  behavior  of  theTRPI  model  is  depicted  in  Figure  1.  The  TPRI  is  plotted  versus 
the  performance  achieved  using  DD  as  a parameter.  The  TPRI  has  a value  of  zero 
when  all  the  performance  measures  have  been  achieved.  The  DD  values  that  are 
between  the  theoretical  bounds  of  zero  and  one  indicate  the  perceived  level  of  difficulty 
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FIGURE  1.  TPRI  PER  ACHIEVED  PERFORMANCE 
WITH  DEGREE  OF  DIFFICULTY  AS  A PARAMETER 


in  developing  the  technology  to  meet  the  required  system  performance  require- 
ments. 

The  theoretical  lower  boundary  of  TPRI  is  identified  at  DD=0,  and  corresponds 
to  the  line  with  the  lowest  TPRI  value.  This  boundary  indicates  that  there  is  no 
perceived  performance  risk  with  guaranteed  success,  and  it  is  equivalent  to 
Garvey’s  approach  of  unmet  performance  as  the  measure  of  system  performance 
risk.  As  the  DD  increases,  the  TPRI  also  increases.  As  expected,  non-zero  DD 
yields  a non-linear  behavior  between  the  achieved  performance  and  the  corre- 
sponding TPRI  value.  The  DD=1.0  provides  the  theoretical  upper  bound,  indi- 
cating a guaranteed  failure.  There  is  a maximum  TPRI  difference  of  0.18  be- 
tween the  two  theoretical  bounded  cases.  The  TPRI  provides  a realistic  measure 
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TABLE  2.  MEASURED  PERFORMANCE  OF  TPMS  PER  TIME  PERIOD 


Measured  Performance,  m 


Technical  Performance  Measures 

fffSI 

t=1 

t=2 

m 

mm 

•..  m ! 

A1  - Average  Processing  Delay  (msec) 

1 

3 

2.86 

1.18 

1.09 

1.03 

A2  - Mean  Time  to  Repair  (minutes) 

10 

50 

43 

43 

27 

12 

9 

A3  - Payload  Weight  (lbs) 

950 

2112 

1764 

1328 

1189 

1008 

948 

A4  - Engagement  Coordination  (sec) 

0.01 

0.1 

0.04 

0.032 

0.02 

0.01 

0.01 

A5  - Interceptors  Available  (#  of  units) 

150 

67 

128 

134 

139 

142 

159 

A6  - Mean  Time  Between  Failure  (hrs) 

500 

100 

189 

223 

348 

379 

521 

A7  - Single  Shot  Success  Probability  (%) 

0.95 

0.87 

0.89 

0.91 

0.934 

0.94 

0.99 

A8  - Damage  Assessment  Accuracy  (%) 

0.995 

0.6 

0.878 

0.94 

.0945 

0.999 

1 

A9  - Software  Coding  (#  coded  modules) 

763 

578 

643 

687 

698 

723 

763 

of  performance  risk  associated  with  the  technology  to  meet,  or  exceed,  the 
established  threshold.  Thus,  the  TPRI  yields  valuable  information  to  support 
the  acquisitionist  and  the  technologist  in  making  sound  decisions  regarding 
the  performance  risk  as  a component  of  the  transition  readiness  of  a technology. 

ANALYSIS 

The  TPRI  is  applied  to  existing  published  data  (Garvey  & Cho,  2003)  as  shown 
in  Table  2.  This  data  consists  of  a set  of  nine  TPMs,  corresponding  threshold  values, 
and  the  measured  performance  over  the  six  various  points  of  time.  An  assessment  of 
the  levels  of  measured  performance  across  each  of  the  TPMs  during  the  time  periods 
is  presented.  For  example,  the  average  processing  delay  increases  in  performance  by 
decreasing  the  average  processing  delay,  measured  in  milliseconds,  from  3 msec  at 
t=l,  to  1.03  msec  at  t=5,  to  exceeding  the  TPM  threshold  (of  1 msec)  with  a measured 
performance  of  0.98  msec  at  the  sixth  time  period.  The  measured  performance  data 
contained  in  Table  2 indicates  each  of  the  nine  TPM  thresholds  are  either  met  or 
exceeded  at  the  sixth  time  period. 


Examples  of  this  decreasing  performance  behavior 
would  include  the  following  four  TPMs:  average 
processing  delays,  mean  time  to  repair,  payload 
weight,  and  engagement  coordination . 


The  achieved  performances  for  TPMs  with  decreasing  performance  behavior  are 
calculated  using  Equation  1 . Examples  of  this  decreasing  performance  behavior  would 


276 


PERFORMANCE -BASED  TECHNOLOGY  ASSESSMENT  METHODOLOGY  TO  SUPPORT  . 'DOD  ACQUISITION 


TABLE  3.  ACHIEVED  PERFORMANCE  (PERCENTAGE) 
OF  TPMS  PER  TIME  PERIOD 


Technical  Performance  Measures 


A1  - Average  Processing  Delay  (msec) 
A2  - Mean  Time  to  Repair  (minutes) 

A3  - Payload  Weight  (lbs) 

A4  - Engagement  Coordination  (sec) 


A5  - Interceptors  Available  (#  of  units) 

A6  - Mean  Time  Between  Failure  (hrs) 
A7  - Single  Shot  Success  Probability  (%) 
A8  - Damage  Assessment  Accuracy  (%) 
A9  - Software  Coding  (#  coded  modules) 


Achieved  Performance,  A 


Threshold  t=1 


0.54  0.72  0,80 

0.10  I 0.25  0.31  0.50  I 1.00 1 1 


150  0.45 

500  0.20 

0.95  0.92  0.94 

0.995  0.60  0.88 

763  0.76  0.84 


include  the  following  four  TPMs:  average  processing  delays,  mean  time  to  repair, 
payload  weight,  and  engagement  coordination.  In  order  to  meet  the  TPM  threshold, 
the  performance  behavior  is  to  be  minimized. 

In  a similar  manner,  Equation  2 is  applied  to  calculate  the  achieved  performances 
for  TPMs  with  increasing  performance  behavior.  Examples  of  this  performance 
behavior  would  include:  number  of  interceptors  available,  mean  time  between 
failure,  single  shot  success  probability,  damage  assessment  accuracy,  and  num- 
ber of  software  modules  written.  The  performance  behavior  of  these  TPMs  must 
be  maximized  to  meet  or  exceed  the  TPM  threshold. 

The  resulting  achieved  performance  data  for  each  of  the  nine  TPMs  and  each 
time  measurement  are  tabulated  in  Table  3.  The  achieved  performance  measure 
is  bounded  by  0 and  1.  The  larger  number  value  of  achieved  performance  in- 
dicates that  the  technology  performance  is  approaching  the  TPM  threshold.  An 
achieved  performance  of  1 indicates  that  the  technology  has  either  met  or  ex- 
ceeded the  TPM  threshold  and  has  accomplished  the  goal  of  entering  the  accept- 
able risk  region. 

Table  4 provides  the  listing  of  the  nine  TPMs  and  the  associated  achieved 
performance.  The  Degree  of  Difficulties  are  arbitrarily  selected  (for  demonstration 
purposes)  for  each  TPM  per  each  of  the  six  time  periods.  The  TPRI  was  computed 
for  each  individual  TPM  requirement  for  each  time  period.  For  example,  at  time  period 
t=l,  the  average  processing  delay  TPM  has  a TPRI  of  0.75,  0.74  at  t=2,  0.19  at  t=3, 
and  continues  to  improve  until  t=6,  when  the  computed  TPRI  is  zero,  indicating  that 
the  TPM  was  met  or  exceeded. 

The  total  system  level  TPRI  is  also  calculated  for  the  technology  for  each  of  the 
six  time  periods.  The  aggregate  TPRI  for  the  technology  is  computed  by  averaging 
the  TPRI  values  of  the  nine  individual  TPM  requirements,  and  is  identified  in  Table 
4.  As  the  technology  advances  in  improved  performance,  the  TPRI  value  decreases, 
indicating  lower  system  performance  risks.  For  example,  the  aggregate  TPRI  for  the 
technology  is  0.64  for  the  first  time  period,  0.49  for  the  second  time  period,  and 
continues  to  improve  (decrease)  through  the  sixth  time  period,  when  the  aggregate 
TPRI  is  zero. 

Figure  2 depicts  the  aggregate  technology-level  TPRI  value  for  each  time 
period.  Observation  of  this  figure  indicates  the  areas  of  significant  risk  and  helps 
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TABLE  4A.  TPMS  AND  ASSOCIATED  ACHIEVED  PERFORMANCE 


Achieved 

Degree  of 

TPRI  per 

Technical  Performance  Measures 

Performance 

Difficulty 

Requirement 

A1  - Average  Processing  Delay  (msec) 

tl  ■ 
0.33 

0.5 

0.75 

A2  - Mean  Time  to  Repair  (minutes) 

0.20 

0.9 

0.88 

A3  - Payload  Weight  (lbs) 

0.45 

0.5 

0.65 

A4  - Engagement  Coordination  (sec) 

0.10 

0.7 

0.94 

A5  - Interceptors  Available  (#  of  units) 

0.45 

0.5 

0.65 

A6  - Mean  Time  Between  Failure  (hrs) 

0.20 

0.9 

0.88 

A7  - Single  Shot  Success  Probability  (%) 

0.92 

0.3 

0.11 

A8  - Damage  Assessment  Accuracy  (%) 

0.60 

0.7 

0.53 

A9  - Software  Coding  (#  coded  modules) 

0.76 

0.7 

0.35 

Total  TPRI,  at  tl 

= 0.64 

Achieved 

Degree  of 

TPRI  per 

Technical  Performance  Measures 

Performance 

Difficulty 

Requirement 

t2 

A1  - Average  Processing  Delay  (msec) 

0.35 

0.5 

0.74 

A2  - Mean  Time  to  Repair  (minutes) 

0.23 

0.9 

0.86 

A3  - Payload  Weight  (lbs) 

0.54 

0.5 

0.56 

A4  - Engagement  Coordination  (sec) 

0.25 

0.5 

0.82 

A5  - Interceptors  Available  (#  of  units) 

0.85 

0.3 

0.18 

A6  - Mean  Time  Between  Failure  (hrs) 

0.38 

0.9 

0.76 

A7  - Single  Shot  Success  Probability  (%) 

0.94 

0.3 

0.08 

A8  - Damage  Assessment  Accuracy  (%) 

0.88 

0.5 

0.17 

A9  - Software  Coding  (#  coded  modules) 

0.84 

0.5 

0.22 

Total  TPRI,  at  t2 

= 0.49 

Achieved 

Degree  of 

TPRI  per 

Technical  Performance  Measures 

Performance 

Difficulty 

Requirement 

A1  - Average  Processing  Delay  (msec) 

13 

0.85 

0.3 

0.19 

A2  - Mean  Time  to  Repair  (minutes) 

0.23 

0.9 

0.86 

A3  - Payload  Weight  (lbs) 

0.72 

0.3 

0.34 

A4  - Engagement  Coordination  (sec) 

0.31 

0.5 

0.77 

A5  - Interceptors  Available  (#  of  units) 

0.89 

0.3 

0.13 

A6  - Mean  Time  Between  Failure  (hrs) 

0.45 

1.9 

0.70 

A7  - Single  Shot  Success  Probability  (%) 

0.96 

1.3 

0.05 

A8  - Damage  Assessment  Accuracy  (%) 

0.94 

0.3 

0.07 

A9  - Software  Coding  (#  coded  modules) 

0.90 

0.3 

0.13 

Total  TPRI,  at  t3  = 0.36 
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TABLE  4B.  TPMS  AND  ASSOCIATED  ACHIEVED  PERFORMANCE  (cont) 


Achieved 

Degree  of 

TPRl  per 

Technical  Performance  Measures 

Performance 

Difficulty 

Requirement 

A1  - Average  Processing  Delay  (msec) 

0.92 

0.1 

0.09 

A2  - Mean  Time  to  Repair  (minutes) 

0.37 

0.7 

0.74 

A3  - Payload  Weight  (lbs) 

0.80 

0.3 

0.25 

A4  - Engagement  Coordination  (sec) 

0.50 

0.5 

0.60 

A5  - Interceptors  Available  (#  of  units) 

0.93 

0.1 

0.08 

A6  - Mean  Time  Between  Failure  (hrs) 

0.70 

0.9 

0.45 

A7  - Single  Shot  Success  Probability  (%) 

0.98 

0.3 

0.02 

A8  - Damage  Assessment  Accuracy  (%) 

0.95 

0.1 

0.05 

A9  - Software  Coding  (#  coded  modules) 

0.91 

0.3 

0.11 

Total  TPRl,  at  t4 

= 0.27 

Achieved 

Degree  of 

TPRl  per 

Technical  Performance  Measures 

Performance 

Difficulty 

Requirement 

b ...7.  « ' .t  **.'>Vx* 

IliSi 

\ ^ ’ f 

A1  - Average  Processing  Delay  (msec) 

0.97 

0.1 

0.03 

A2  - Mean  Time  to  Repair  (minutes) 

0.83 

0.7 

0.25 

A3  - Payload  Weight  (lbs) 

0.94 

0.3 

0.07 

A4  - Engagement  Coordination  (sec) 

1.00 

0 

0.00 

A5  - Interceptors  Available  (#  of  units) 

0.95 

0.1 

0.06 

A6  - Mean  Time  Between  Failure  (hrs) 

0.76 

0.9 

0.38 

A7  - Single  Shot  Success  Probability  (%) 

0.99 

0.1 

0.01 

A8  - Damage  Assessment  Accuracy  (%) 

1.00 

0 

0.00 

A9  - Software  Coding  (#  coded  modules) 

0.95 

0.3 

0.07 

Total  TPRl.  at  t5 

= 0.10 

Achieved 

Degree  of 

TPRl  per 

Technical  Performance  Measures 

Performance 

Difficulty 

Requirement 

If 

v'/f  ’C  V ; 7.  'f. 

A1  - Average  Processing  Delay  (msec) 

1 

0 

0.00 

A2  - Mean  Time  to  Repair  (minutes) 

1 

0 

0.00 

A3  - Payload  Weight  (lbs) 

1 

0 

0.00 

A4  - Engagement  Coordination  (sec) 

1 

0 

0.00 

A5  - Interceptors  Available  (#  of  units) 

1 

0 

0.00 

A6  - Mean  Time  Between  Failure  (hrs) 

1 

0 

0.00 

A7  - Single  Shot  Success  Probability  (%) 

1 

0 

0.00 

A8  - Damage  Assessment  Accuracy  (%) 

1 

0 

0.00 

A9  - Software  Coding  (#  coded  modules) 

1 

0 

0.00 

Total  TPRl,  at  t6  = 0.0 
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Technology  Performance  Risk  Index  (TPRI) 


Time  period 

FIGURE  2.  SYSTEM-LEVEL  TECHNOLOGY  PERFORMANCE 
RISK  INDEX  PER  TIME  PERIOD 


decision  makers  with  providing  additional  information  pertaining  to  the  perfor- 
mance risk  involved  with  a technology  to  meet  the  threshold  values  associated 
with  TPMs.  At  the  sixth  time  period,  the  technology  has  achieved  the  acceptable 
risk  region  and  has  a zero-valued  TPRI. 


TPRI  BENEFITS 

The  TPRI  provides  a realistic  performance  risk  assessment  based  on  performance 
requirements,  Degree  of  Difficulty,  and  unmet  performance.  These  components  are 
combined  in  a closed-loop  feedback  manner  to  calculate  the  technology  performance 
risk.  This  decision  support  tool  provides  insight  to  the  risk  involved  in  the  unmet 
performance  to  meet  TPM  thresholds,  and  the  level  of  activity  required  to  meet  or 
exceed  the  threshold.  TPRI  applies  performance  risks  associated  with  an  unmet  re- 
quirement as  correction/feedback  to  the  achieved  performance.  Since  TPRI  is  based 
on  meeting  TPM  thresholds  and  identifying  associated  risks  with  unmet  requirement, 
it  provides  common  ground  between  technologist  and  acquisitionist.  As  a result,  an 
improved  understanding  of  the  technology’s  capabilities  to  support  the  acquisition 
system  is  gained. 


CONCLUSION 


This  research  has  focused  on  the  development  and  demonstration  of  a quantitative 
methodology  to  evaluate  and  monitor  technology  to  determine  transition  readiness. 
The  TPRI  provides  a means  to  assess  potential  technologies  and  assist  the  decision 
maker  in  where  to  apply  resources  to  address  unmet  requirements.  The  TPRI  supports 
monitoring  of  performance  of  a technology  against  threshold  limits.  It  integrates 
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the  technology  degree  of  difficulty  along  with  the  system  unmet  requirements 
in  a closed  loop  to  gain  additional  information  regarding  a technology’s  perfor- 
mance risk  over  time.  Additionally,  it  reduces  the  probability  associated  with 
immature  technology  being  transitioned  to  a weapon  system  prematurely  This 
approach  is  anticipated  to  contribute  toward  level  of  success  pertaining  to  the 
integration  of  technology  into  system(s)  of  the  aerospace  and  defense  domains. 


NEXT  STEPS 

Future  efforts  will  include  examining  methods  to  combine  this  quantitative 
Technology  Performance  Risk  Index  (TPRI)  with  a technology  maturity  metric, 
such  as  the  widely  utilized  Technology  Readiness  Level  (TRL).  Various  ranking 
and  weighting  schemes  are  planned  to  be  examined  for  potential  applicability  in 
the  TPRI  calculation.  There  are  plans  to  apply  the  TPRI  as  a decision  support 
tool  to  assist  a decision  maker  with  evaluating  and  selecting  the  best  of  identi- 
fied multiple  technologies  across  same  set  of  TPMs. 
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